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INTRODUCTION 


Our  highly  innovative  project  was  designed  to  identify  more  precise  and  objective,  performance-based 
measures  of  the  Broader  Autism  Phenotype  (BAP)  to  advance  scientific  investigation  of  the  causes  and 
treatment  of  Autism  Spectrum  Disorders  (ASD).  The  BAP  is  a  milder  phenotypic  variant  of  ASD  that 
primarily  involves  social-relatedness  difficulties.  Family  members  with  the  BAP  are  likely  to  be  carriers  of 
one  or  more  ASD  susceptibility  genes,  providing  an  ideal  model  for  identifying  genes  underpinning 
impaired  social  behavior.  Thus,  our  study  focused  on  the  social  communication  difficulties  commonly 
experienced  by  people  with  the  BAP  using  new  and  sophisticated  techniques  that  we  developed  to  capture 
the  acoustic  properties  of  speech  important  for  emotional  expression.  We  investigated  this  issue  in  detail, 
both  in  individuals  with  and  without  the  BAP,  using  a  range  of  tasks  measuring  skills  important  in 
emotional  and  social  interactions.  This  project  also  maps  the  brain  systems  that  underpin  social 
communication  in  the  BAP  by  using  state-of-the-art  brain  imaging  techniques  that  measure  brain  structure 
and  function.  Our  study  design  included  linking  these  behavioral  and  brain  findings  to  gain  a  more 
complete  understanding  of  the  BAP. 

Our  novel  approach  involved  assessing  people  from  rare  large  families,  in  which  there  are  many  members 
with  either  autism  or  the  BAP  with  these  social  communication  difficulties.  This  allows  complex  genetic 
patterns  to  be  traced  in  these  families,  so  that  we  can  assess  how  the  social-relatedness  difficulties 
associated  with  autism  and  the  BAP  are  inherited  across  generations.  This  is  a  highly  effective  way  to  find 
dominant  genes  causing  social  difficulties  in  autism  and  the  BAP.  In  this  way,  the  current  project  not  only 
changes  how  autistic  traits  are  viewed,  as  falling  along  a  spectrum  that  blends  with  the  normal  population, 
but  it  also  provides  new  neuroscientific  insights  into  the  changes  in  the  brain  that  underpin  autistic 
behavior.  This  project  therefore  lays  the  vital  foundations  for  the  development  of  targeted  treatments 
specifically  designed  to  ease  the  profound  social  difficulties  than  can  be  experienced  by  people  with  ASD 
and  their  families. 


Keywords:  Autism  Spectrum  Disorders  (ASD),  Broader  Autism  Phenotype  (BAP),  nonverbal  emotions, 
social  cognition,  structural  brain  imaging,  functional  brain  imaging,  endophenotype 
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OVERALL  PROJECT  SUMMARY 


The  research  objectives  of  our  project  included  performing  a  detailed  behavioral  analysis  of  emotional 
identification,  imitation,  and  expression  using  our  new  audiovisual  technologies  to  fully  characterize  the 
psychoacoustic  endophenotype  of  the  BAP.  We  also  linked  this  endophenotypic  profile  to  measures  of 
pragmatic  language,  empathy  and  social  cognition.  We  then  used  advanced  functional  and  structural 
neuroimaging  to  investigate  differences  in  the  neural  correlates  of  the  perception  and  production  of 
emotionally  expressive  speech  in  BAP  participants  and  healthy  controls,  with  a  view  to  identifying  neural 
endophenotypic  markers,  and  determining  the  relationship  between  the  neural  and  psychoacoustic 
endophenotypes,  as  well  as  their  sensitivity  and  specificity  to  the  BAP.  By  addressing  these  objectives 
our  project  provides  clinicians  with  innovative  neurobehavioral  assessment  tools  for  precise 
characterization  of  the  BAP  that  complement  existing  clinical  methods  for  diagnosis,  as  well  as 
contributing  to  our  understanding  of  the  neurobiological  basis  of  emotional  expression  in  the  BAP,  ultimately 
contributing  to  molecular  genetic  analyses  to  discover  ASD  genes. 

In  light  of  these  objectives,  the  information  below  describes  our  research  accomplishments,  listed 
according  to  each  milestone  described  in  our  approved  Statement  of  Work  (effective  start  date:  April  19, 
2013).  Methodology  for  our  behavioral  and  functional  neuroimaging  paradigms  is  also  presented,  along  with 
a  summary  of  our  findings. 


Milestone  #1;  Regulatory  review  and  approval  for  human  study 

Local  IRB  (Royal  Children’s  Hospital,  Human  Research  Ethics  Committee)  approval  to  conduct  the  study 
was  granted  on  July  23,  2012.  Reports  for  annual  continuing  review  were  subsequently  submitted  and 
approved  by  our  local  IRB  between  2012-2014. 


Milestone  #2:  Protocol  and  technique 

An  interactive  web-based  computer  program  was  purpose-built  and  then  extensively  piloted  to  collect 
data  for  the  specific  tasks  included  in  our  behavioral  paradigm  to  measure  social  communication 
difficulties  in  the  BAP.  Among  other  things,  this  web-based  program  incorporates  a  range  of  vocal  sound 
files  of  varying  nonverbal  emotions  to  which  participants  respond.  It  also  uses  highly  innovative, 
purpose-built  voice  recording  software  to  collect  participant  samples  of  emotions.  It  has  been  effectively 
applied  to  collect  behavioral  data  from  study  participants,  with  this  data  now  undergoing  final  analyses 
using  purpose-developed  computational  algorithms  to  characterize  the  psychoacoustic  endophenotype  of 
the  BAP.  This  data  is  targeted  for  publication  in  a  high  impact,  international  scientific  journal  over  the 
next  12  months.  The  novel  web-based  methodology  developed  for  our  behavioral  paradigm  and  the  findings 
from  this  work  to  date  are  described  on  pages  6-10  of  this  report. 

A  set  of  in-scanner  fMRI  tasks  that  produce  behavioral  responses  equivalent  to  the  out-of-scanner 
behavioral  paradigm  was  also  purpose-developed  and  extensively  piloted.  This  ensured  that  patterns  of 
brain  activation  collected  in-scanner  can  be  accurately  correlated  with  out-of-scanner  behavioral 
performance  to  ensure  cognitive  and  emotional  processes  of  interest  are  effectively  captured.  The 
paradigm  employs  state-of-the-art  structural  and  functional  MRI  techniques,  for  which  our  research  team 
is  recognized  as  leading  the  field.  Combined,  the  behavioral  and  neuroimaging  paradigms  represent 
sophisticated  research  tools  that  have  been  successfully  used  with  the  study  participants.  Analyses  of  the 
neuroimaging  data  as  well  as  statistical  correlations  between  the  behavioral  and  neuroimaging  data  are 
currently  underway,  and  are  again  targeted  for  publication  in  high  impact,  international  scientific  journals 
over  the  next  12  months.  The  novel  methodology  developed  for  our  functional  neuroimaging  paradigm  and 
the  findings  from  this  work  to  date  are  described  on  pages  1 1-16  of  this  report. 
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Milestone  #3:  Participant  ascertainment 


As  planned,  family  members  from  the  rare  large  families  with  multiple  individuals  diagnosed  with  either 
ASD  or  the  BAP  have  been  assessed  with  the  Family  History  Interview  as  well  as  an  extensive  range  of 
cognitive  and  behavioral  measures  to  determine  BAP  status.  The  social  discourse  of  these  family  members 
has  also  been  precisely  analyzed  using  purpose-written  auditory  software,  to  identify  individuals  with  and 
without  the  psychoacoustic  marker  of  the  BAP.  Control  participants  matched  for  age,  sex  and  IQ  with  no 
history  of  neurologic,  psychiatric  or  hearing  disorders  have  also  been  successfully  identified  and  assessed 
with  the  study  measures. 


Milestone  #4:  Publication  of  behavioral  findings  and  presentation  of  results  at  international 

conferences 


Behavioral  data  has  been  successfully  collected  from  project  participants  using  our  purpose-built  web- 
based  computer  program  incorporating  various  emotion-related  tasks,  including  stimuli  from  the  Montreal 
Affective  Voices  (MAV;  Belin,  Fillion-Bilodeau,  &  Gosselin,  2011).  Analyses  of  the  behavioral  data  using 
IBM  SPSS  Statistics  are  being  finalized,  with  published  outcomes  expected  within  the  next  12  months, 
targeting  high  impact  scientific  journals.  Conference  abstracts  will  also  be  submitted  to  national  and 
international  community  and  scientific  meetings,  including  the  International  Meeting  for  Autism 
Research  (IMFAR),  to  ensure  that  our  behavioral  findings  are  widely  disseminated.  For  all  publications 
and  presentations  the  funding  support  of  the  US  Department  of  Defense  will  be  appropriately  recognized 
and  acknowledged. 

Behavioral  Methodology  and  Findings 

Our  web-based  computer  program  (URL:  http://emotions.sollysweb.com/index_new.php)  incorporated  a  range 
of  emotion-related  tasks  to  measure  our  participants’  ability  to  spontaneously  produce,  recognize  and  imitate 
nonverbal  vocal  emotional  expressions  belonging  to  the  six  basic  emotions  (i.e.,  anger,  fear,  disgust,  surprise, 
happiness,  sadness).  Other  measures  of  general  auditory  processing  ability  were  also  included  in  our  program  to 
account  for  individual  differences  in  pitch  perception,  which  may  confound  performance  on  the  emotion-related 
tasks.  To  maintain  participant  motivation,  throughout  the  program  participants  were  presented  with  a  “progress 
bar/circle”  to  keep  track  of  the  number  of  items/tasks  they  were  yet  to  complete.  Participants  completed  all 
tasks  alone  in  an  anechoic  chamber.  Voice  recordings  were  collected  in  WAV  format  (sampling  frequency: 
44.1  kHz)  using  a  high  quality  digital  microphone,  and  auditory  stimuli  were  presented  from  an  external 
subwoofer  speaker  at  a  comfortable  volume  (sound  intensity:  approximately  70  dB).  All  data  collected  with  this 
computer  program  were  stored  on  a  secure,  password-protected  web-based  server. 

Emotion  Production  Task 

Figure  1  shows  a  typical  trial  from  the  Emotion  Production  Task.  In  each  trial,  participants  were  presented  with 
one  basic  emotion  to  express  with  their  voice.  Of  note,  participants  were  first  asked  to  imagine  that  they  were 
feeling  the  emotion  before  expressing  it  vocally.  Across  trials,  participants  were  explicitly  advised  not  to  use 
words  (i.e.,  nonverbal)  to  express  each  emotion  (e.g.,  “No!”  “Hey!”  “Wow!”  “Yay!”)  in  order  to  minimize  the 
presence  of  interactions  between  emotional  and  linguistic  properties  of  such  vocalizations  when  analyzing 
acoustic  markers  in  the  data. 
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Activity  1 


EXPRESS  THE  EMOTION 


FEAR 


Imagine  you  are  afraid. 

Then,  click  the  record  button  below,  wait  2  seconds  and  express  your  FEAR  one  time 
(without  using  words).  Click  the  button  again  when  you  are  done. 


P 


Task  1  of  18 


Figure  1.  Example  of  a  typical  trial  in  the  Emotion  Production  Task  from  the  web-based  computer  program.  In  this  trial,  the  participant  is 
presented  with  the  basic  emotion  of  “Fear”  and  instructed  to  imagine  that  they  are  feeling  afraid  before  producing  a  fearful  expression  with  their 
voice.  The  participant  then  clicks  the  “Record”  button  to  record  their  expression  and  can  refer  to  the  black  bar  to  monitor  microphone  levels.  The 
progress  bar  at  the  top  of  the  screen  indicates  that  the  participant  has  five  additional  tasks  to  complete  (mix  of  emotion-related  tasks  and  tasks  of 
general  sound  processing),  while  the  progress  circle  at  the  bottom  shows  the  number  of  trials  remaining  (17)  in  this  particular  task. 


Participants  completed  a  total  of  18  trials,  over  which  they  were  required  to  express  the  same  basic  emotion 
across  three  consecutive  trials.  Data  from  this  task  is  being  analyzed  with  purpose-built  statistical  algorithms  to 
determine  the  specific  acoustic  properties  of  the  six  basic  emotions.  Figure  2  shows  waveforms  and 
spectrograms  of  nonverbal  vocal  emotional  expressions  belonging  to  each  of  the  six  basic  emotions,  as 
produced  by  male  and  female  participants. 


Figure  2.  Waveforms  (top  image)  and  spectrograms  (bottom  panel)  from  spontaneous  nonverbal  vocal  emotional  expressions  belonging  to  the  six 
basic  emotions,  as  produced  by  male  and  female  participants  using  the  web-based  Emotion  Production  Task. 
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Emotion  Recognition  Task 

Figure  3  shows  a  typical  trial  from  the  Emotion  Recognition  Task.  In  each  trial,  participants  were  first 
presented  with  a  vocal  stimulus  from  the  Montreal  Affective  Voices  (MAV)  task,  then  asked  to  identify  the 
emotion  in  the  voice,  and  to  rate  its  intensity  (l=weak  emotion  to  7=strong  emotion).  MAV  stimuli  with  no 
emotional  content  (i.e.,  neutral  voices,  rating  of  0)  were  also  presented  as  a  control  task.  Participants  were 
required  to  make  their  responses  by  clicking  along  the  appropriate  dimension  of  the  visual  wheel  shown  in 
Figure  3,  or  to  click  on  the  central  circle  (i.e.,  “No  Emotion”  category)  to  identify  neutral  voices. 


WHAT  IS  THE  EMOTION 


Click  ©  to  hear  someone  express  an  emotion 


You  may  listen  to  the  i /oice  as  many  times  as  you  like 


Think  about  the  type  of  emotion  expressed  in  their  voice.  Then,  rate  how  strong  the  emotion  sounds  by 
clicking  on  a  number  in  the  line,  from  1  (weak  emotion)  to  7  (strong  emotion).  If  no  emotion  is  expressed 
in  their  voice,  then  click  on  the  circle  in  the  middle. 

DISGUST 


Task  1  of  42 


Click  to  proceed 


Figure  3.  Example  of  a  typical  trial  in  the  Emotion  Recognition  Task  from  the  web-based  computer  program.  Participants  are  instructed  to  click 
on  the  green  button  to  hear  a  vocal  stimulus  (MAV),  and  to  then  identify  the  emotion  in  the  voice  and  rate  its  intensity  using  the  visual  wheel. 
Here,  the  participant  has  clicked  on  the  number  “6”  on  the  “ANGER”  arm  of  the  wheel  to  indicate  that  the  MAV  stimulus  represents  a  relatively 
strong  angry  expression.  The  progress  bar  at  the  top  of  the  screen  indicates  that  the  participant  has  four  additional  tasks  to  complete  (mix  of 
emotion-related  tasks  and  tasks  of  general  sound  processing),  while  the  progress  circle  at  the  bottom  shows  the  number  of  trials  remaining  (41)  in 
this  particular  task.  Once  completed,  the  participant  then  clicks  the  blue  arrow  to  proceed  to  the  next  trial  of  the  task. 


Participants  completed  a  total  of  42  randomized  trials,  over  which  6  MAV  stimuli  (3  male,  3  female  voices) 
were  presented  from  each  of  the  six  basic  emotions  and  the  “No  Emotion”  category.  Figure  4  shows  the 
distributions  of  accuracy  scores  and  intensity  ratings  from  the  Emotion  Recognition  Task  from  a  sample  of 
control  participants.  As  shown,  the  majority  of  control  participants  achieved  high  scores  for  percentage 
accuracy  (M=  80.80,  &D=7.385),  while  more  variation  in  the  spread  of  control  data  was  found  for  intensity 
ratings  (M=137.18,  SD= 24.5). 
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Figure  4.  Histograms  of  the  distribution  of  a)  total  percentage  accuracy  scores  and  b)  total  intensity  ratings  from  a  sample  of  control  participants 
performing  the  web-based  Emotion  Recognition  Task. 


Emotion  Imitation  Task 

Figure  5  shows  a  typical  trial  from  the  Emotion  Imitation  Task.  In  each  trial,  participants  were  presented  with  a 
vocal  stimulus  from  the  Montreal  Affective  Voices  (MAV)  task  and  informed  about  the  type  of  emotion 
expressed  in  the  voice.  Participants  were  then  required  to  imitate  the  speaker’s  emotional  expression  as 
accurately  as  possible.  Gender-specific  MAV  stimuli  were  presented  to  each  participant  (e.g.,  male  voices  to 
male  participants)  to  account  for  differences  in  acoustic  properties  across  male  and  female  voices. 


COPY  THE  EMOTION 


Click  hear  someone  express 

HAPPINESS 

_ 0 _ 

Then,  click  the  record  button  below,  wait  for  2-seconds  and  copy  the  emotion  once  with 
your  voice  as  best  as  you  can. 

Click  the  button  again  when  you  are  done. 


Record 


20  10  7  5  3  0  3  5 


Task  1  of  35 


Figure  5.  Example  of  a  typical  trial  in  the  Emotion  Imitation  Task  from  the  web-based  computer  program.  In  this  trial,  a  participant  is  instructed 
to  click  the  green  button  to  hear  a  happy  vocal  expression  (MAV  stimulus)  and  to  then  imitate  the  happy  expression  as  accurately  as  possible.  As 
per  the  Emotion  Production  Task,  participants  then  click  the  “Record”  button  to  record  their  expression  and  can  refer  to  the  black  bar  to  monitor 
microphone  levels.  The  progress  bar  at  the  top  of  the  screen  indicates  that  the  participant  has  two  additional  tasks  to  complete  (mix  of  emotion- 
related  tasks  and  tasks  of  general  sound  processing),  while  the  progress  circle  at  the  bottom  shows  the  number  of  trials  remaining  (34)  in  this 
particular  task. 
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Participants  completed  a  total  of  35  randomized  trials,  over  which  five  gender-specific  MAV  stimuli  were 
presented  from  each  of  the  six  emotions  and  the  “No  Emotion”  category.  Data  from  this  task  are  being  analyzed 
with  purpose-built  statistical  algorithms,  which  compare  specific  acoustic  properties  between  the  participants’ 
vocal  expressions  and  vocal  expressions  from  the  MAV  task  to  determine  the  accuracy  of  imitation.  Figure  6 
shows  waveforms  and  spectrograms  of  MAV  stimuli  templates  belonging  to  each  of  the  six  basic  emotions, 
along  with  imitations  made  by  a  male  subject  participating  in  the  study. 


e)  HAPPINESS  f)  SADNESS 


Jsl  I  i  k 


Figure  6.  Waveforms  (top  panel)  and  spectrograms  (bottom  panel)  of  male  MAV  stimuli  for  each  of  the  six  basic  emotions  and  the  corresponding 
vocal  responses  produced  by  a  single  male  participant  performing  the  web-based  Emotion  Imitation  Task.  Here,  MAV  stimuli  and  their 
corresponding  imitations  are  combined  into  a  single  sound  file  for  each  of  the  six  basic  emotions. 


Milestone  #5:  Publication  of  structural  and  functional  neuroimaging  results  and  presentation  of 

results  at  international  conference 


Neuroimaging  data  has  been  successfully  collected  from  project  participants  using  our  state-of-the-art 
functional  and  structural  techniques.  Analyses  of  the  neuroimaging  data  using  both  statistical  parametric 
mapping  (SPM)  software  and  our  in-house  techniques  are  currently  being  finalized,  with  published 
outcomes  expected  within  the  next  12  months  targeting  high  impact  scientific  journals.  Published 
outcomes  will  include  the  relationship  between  neural  and  psychoacoustic  endophenotypes  of  the  BAP,  as 
well  as  their  sensitivity  and  specificity  to  the  BAP.  These  findings  will  also  be  submitted  for  presentation 
at  national  and  international  community  and  scientific  meetings,  including  the  International  Meeting  for 
Autism  Research  (IMFAR).  Again,  for  all  publications  and  presentations  the  funding  support  of  the  US 
Department  of  Defense  will  be  appropriately  recognized  and  acknowledged. 
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Functional  Neuroimaging  Methodology  and  Findings 


Participants  performed  two  tasks  of  emotion  processing  within  a  3-Tesla  Siemens  MRI  scanner.  These  tasks 
were  designed  to  identify  activity  in  brain  regions  underpinning  the  production  and  recognition  of  nonverbal 
vocal  emotional  expressions  belonging  to  the  six  basic  emotions  (i.e.,  anger,  disgust,  fear,  happiness,  sadness, 
surprise).  Participants  received  prior  training  for  the  in- scanner  tasks  on  a  computer  program  in  a  mock 
scanner  environment  to  increase  the  reliability  of  the  results.  The  brain  images  presented  below  (Figures  9  & 
12)  show  that  the  functional  neuroimaging  paradigm  successfully  activated  key  brain  regions  associated  with 
emotion  processing,  including  the  hypothesized  mirror  neuron  system  (MNS),  which  underlies  social 
cognitive  skills  such  as  the  ability  to  empathize  with  others.  As  described  in  our  initial  project  proposal,  our 
neuroimaging  paradigm  has  therefore  successfully  allowed  associations  between  MNS  functioning  and  social 
communication  ability  to  be  examined,  contributing  important  scientific  knowledge  about  the  neurobiology 
and  genetic  basis  of  social  behavior  in  ASD. 

Task  1:  Vocal  Emotion  Production 

Whilst  in-scanner,  participants  were  required  to  make  nonverbal  vocalizations  belonging  to  two  primary 
categories:  (1)  vocal  expressions  of  the  six  basic  emotions  (Emotion  Condition),  and  (2)  non-emotional 
vocalizations  (Neutral  Condition).  Figure  7  shows  the  sequence  of  events  in  a  typical  trial  from  each 
condition.  In  the  Emotion  Condition,  emotion  labels  (i.e.,  “Anger”,  “Disgust”,  “Fear”,  “Happiness”, 
“Sadness”,  “Surprise”)  were  presented  visually  across  trials.  Participants  were  instructed  to  imagine  that  they 
were  feeling  the  emotion  presented  and  then  to  express  the  emotion  vocally  without  using  words.  In  the 
Neutral  Condition,  the  label  “No  Emotion”  was  presented  visually  across  trials,  and  participants  were 
instructed  to  produce  a  stereotyped  monotonous  vocal  response,  which  they  had  practised  with  the 
experimenter  prior  to  scanning.  Across  conditions,  participants  were  instructed  and  had  practised  producing 
vocalizations  with  minimal  facial  and  body  movement. 


(a) 


(b) 


Figure  7.  Sequence  of  events  in  a  single  trial  from  the  (a)  Emotion  Condition  and  (b)  Neutral  Condition  in  Task  1.  In  the  Emotion  Condition 
trial  shown  here,  the  emotion  label  “Anger”  is  presented  for  3s,  during  which  the  participant  is  required  to  imagine  feeling  angry  and  to  make 
an  angry  vocal  expression.  After  the  offset  of  the  label,  a  fixation  cross  is  presented  for  5s.  In  the  Neutral  Condition  trial  shown  here,  the  label 
“No  Emotion”  is  presented  for  3s,  during  which  the  participant  is  required  to  make  a  stereotyped  monotonous  vocalization,  void  of  emotional 
content.  A  fixation  cross  is  then  presented  for  5s. 
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Task  1  was  designed  as  an  event-related  paradigm  (Figure  8)  to  allow  volumes  containing  significant  motion 
artefacts  to  be  excluded  from  analysis.  A  total  of  72  trials  were  included  in  the  task,  with  36  trials  from  each 
condition.  Trials  belonging  to  the  Emotion  Condition  were  presented  in  groups  of  three,  whereby  the  same 
type  of  emotion  label  was  presented  across  trials  (i.e.,  participants  were  expected  to  make  three  consecutive 
expressions  of  the  same  basic  emotion,  allowing  them  to  become  fully  absorbed  in  the  experience  of 
producing  the  emotion).  Trials  belonging  to  the  Neutral  Condition  were  also  presented  in  groups  of  three. 
Each  group  of  three  trials  alternated  between  conditions.  A  total  of  216  whole  brain  volumes  were  acquired 
for  this  fMRI  run  using  a  whole-brain  T2*-weighted  gradient-recalled  echo-planar  imaging  sequence  (EPI) 
with  a  TR  of  3s. 


I  I 

I  I 

I  I 

J_ L 


"ANGER"  Trials 


nr 

"NO  EMOTION"  Trials 


I  I 

I  I 

I  I 
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"DISGUST  Trials 


"NO  EMOTION"  Trials 


Figure  8.  Series  of  events  representing  a  sample  of  single  trials  from  Task  1.  Three  trials  from  the  Emotion  Condition  were  presented 
consecutively,  followed  by  three  consecutive  trials  from  the  Neutral  Condition.  Across  each  group  of  three  trials  from  the  Emotion  Condition, 
participants  were  expected  to  make  emotional  expressions  belonging  to  the  same  emotion  category  (e.g.,  “Anger”,  “Disgust”).  A  fixed  inter¬ 
stimulus  interval  of  9s  was  employed  between  trials. 


Results  to  date  show  that  the  contrast  of  blood  oxygenated  level  dependent  (BOLD)  signal  acquired  during 
the  Emotion  Condition  versus  Neutral  Condition  revealed  several  regions  of  significantly  greater  neuronal 
activation  (t>3. 16,  pcO.OOl)  associated  with  the  production  of  vocal  emotional  expressions  compared  to  non- 
emotional  vocalizations.  These  regions  include:  (a)  bilateral  inferior  frontal  gyri,  (b)  bilateral  superior  frontal 
gyri  (including  premotor  cortex,  BA6),  (c)  bilateral  superior  temporal  gyrus  (including  temporal  pole,  BA38), 
(d)  left  insula,  (e)  left  anterior  cingulate  gyrus,  (f)  left  parietal  lobe,  (g)  right  caudate,  and  (h)  bilateral 
cerebellum.  A  number  of  these  regions  are  displayed  in  Figure  9,  which  shows  a  single  subject’s  BOLD 
activation  map  from  Task  1. 


Of  note,  our  findings  are  consistent  with  other  neuroimaging  studies  that  have  suggested  that  the  regions  of 
activation  targeted  in  Task  1  underlie  various  emotion-related  constructs.  For  example,  limbic  structures  like 
the  anterior  cingulate  cortex  and  insula  have  been  associated  with  emotional  processes  that  involve  high 
cognitive  demand  (e.g.,  emotion  recognition),  the  recall  of  emotional  memories  and  emotional  imagery  (see 
Phan,  Wager,  Taylor,  &  Liberzon,  2002).  In  addition,  the  superior  frontal  gyrus,  middle  frontal  gyrus  and 
cerebellum  have  been  implicated  in  studies  on  emotion  regulation  (Beauregard,  Levesque,  &  Bourgouin, 
2001;  Eippert  et  al.,  2007;  Schutter  &  Honk,  2009).  Further,  the  inferior  frontal  gyrus  has  been  regarded  as  a 
voice-selective  region  that  is  particularly  sensitive  to  nonverbal  vocal  emotional  expressions  (Fecteau, 
Armony,  Joanette,  &  Belin,  2005).  Interestingly,  the  inferior  frontal  gyrus — along  with  the  superior  temporal 
gyrus  and  parietal  lobe  also  activated  here — is  a  key  structure  in  the  MNS,  which  has  important  implications 
for  understanding  social  cognition  and  empathy  in  the  BAP  (see  Iacoboni  &  Dapretto,  2006),  as  discussed 
further  in  our  General  Discussion  below. 
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Figure  9.  Activation  map  of  axial  slices  from  a  single  subject  showing  contrast  in  BOLD  signal  acquired  during  the  Emotion  Condition  versus 
Neutral  Condition  in  Task  1  (t>3. 16,  p<0.001).  In  this  subject,  several  regions  of  significantly  greater  neuronal  activation  were  associated  with  the 
production  of  vocal  emotion  expressions,  including:  a)  superior  frontal  gyrus,  b)  right  middle  frontal  gyrus,  c)  anterior  cingulate,  d)  left  post- 
central  gyrus  (parietal  lobe),  e)  right  caudate,  f)  left  inferior  frontal  gyrus,  g)  left  insula,  h)  right  insula,  i)  right  middle  temporal  gyrus,  and  j)  right 
cerebellum. 


Task  2:  Vocal  Emotion  Recognition 

In  this  task,  participants  listened  to  male  and  female  nonverbal  vocalizations  belonging  to  two  primary 
categories:  (1)  vocal  expressions  of  the  six  basic  emotions  (Emotion  Condition),  and  (2)  non-emotional 
vocalizations  (Neutral  Condition).  Figure  10  shows  the  sequence  of  events  in  the  first  trial  from  each 
condition.  All  vocal  stimuli  were  taken  from  the  Montreal  Affective  Voices  (MAV).  In  the  Emotion 
Condition,  participants  were  required  to  identify  the  emotion  in  the  speaker’s  voice  by  making  a  forced- 
choice  discrimination  between  two  emotion  labels  presented  visually  (e.g.,  “Anger”  and  “Disgust”).  In  the 
Neutral  Condition,  participants  were  required  to  identify  the  gender  of  the  speaker  by  making  a  forced-choice 
discrimination  between  the  labels  “Male”  and  “Female”  presented  visually.  Across  both  conditions, 
participants  pressed  a  button  box  with  their  middle  or  index  finger  to  indicate  their  response. 
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Figure  10.  Sequence  of  events  in  the  first  trial  of  a  block  from  the  (a)  Emotion  Condition  and  (b)  Neutral  Condition  in  Task  2.  At  the  onset  of 
a  block  from  the  Emotion  Condition,  the  label  “Emotion”  is  presented  for  1.5s.  In  the  Emotion  trial  shown  here,  a  MAV  stimulus  of  an  angry 
female  voice  is  then  presented  (duration  of  MAV  stimuli  vary  across  trials).  Next,  the  emotion  labels  “Anger”  and  “Disgust”  are  presented  for 
2s,  during  which  the  participant  is  required  to  select  the  emotion  of  the  speaker  by  pressing  a  button  with  the  index  finger  (for  top  label;  correct 
response  for  this  trial)  or  middle  finger  (for  bottom  label).  At  the  onset  of  a  block  from  the  Neutral  Condition,  the  label  “Gender”  is  presented 
for  1.5s.  In  the  Neutral  trial  shown  here,  a  MAV  stimulus  of  a  non-emotional,  monotonous  female  voice  is  then  presented.  The  gender  labels 
“Male”  and  “Female”  are  then  presented  for  2s,  during  which  the  participant  is  required  to  select  the  gender  of  the  speaker  by  pressing  a  button 
with  the  index  finger  (for  top  label)  or  middle  finger  (for  bottom  label;  correct  response  for  this  trial). 


Task  2  was  designed  as  an  Epoch-based  block  paradigm  (Figure  11).  A  total  of  12  blocks  were  included  in 
the  task,  with  6  blocks  (6  trials  per  block)  for  each  condition.  The  label  “Emotion”  was  presented  at  the  onset 
of  each  block  from  the  Emotion  Condition  to  alert  participants  to  identify  the  speaker’s  emotion  in  the  MAV 
stimuli  presented  across  all  trials  within  that  block  (Figure  10).  Conversely,  the  label  “Gender”  was  presented 
at  the  onset  of  each  block  from  the  Neutral  Condition,  to  alert  participants  to  identify  the  speaker’s  gender  in 
the  MAV  stimuli  presented  across  all  trials  within  that  block  (Figure  10).  Blocks  were  alternated  between 
conditions,  and  trials  within  each  block  were  presented  in  pseudo-randomized  order.  A  total  of  121  whole 
brain  volumes  were  acquired  for  this  fMRI  run  using  a  whole-brain  T2* -weighted  gradient-recalled  EPI  with 
a  TR  of  3s. 


"Emotion"  block 

(6  trials  per  block;  Mock  duration:  30s] 


"Neutral"  block 

<6  trials  per  block;  block  duration:  30s) 


Figure  11.  Classic  “boxcar”  model  for  Task  2  representing  epochs  of  sustained  activation  for  blocks  from  the  Emotion  Condition  relative  to 
blocks  from  the  Neutral  condition.  Blocks  for  the  Emotion  Condition  were  alternated  with  blocks  for  the  Neutral  Condition,  with  6  trials 
presented  in  each  block  over  a  30s  duration. 
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Our  findings  to  date  indicate  that  the  contrast  of  BOLD  signal  acquired  during  the  Emotion  Condition  versus 
the  Neutral  Condition  revealed  several  regions  of  significantly  greater  neuronal  activation  (t>3. 17,  pcO.OOl) 
associated  with  the  recognition  of  vocal  emotional  expressions  compared  to  non-emotional  vocalizations. 
Specifically,  these  regions  include:  (a)  bilateral  superior  frontal  gyri  (including  premotor  cortex,  BA6),  (b) 
bilateral  inferior  frontal  gyri,  (c)  bilateral  middle  frontal  gyri,  (d)  bilateral  superior  temporal  gyri  (including 
temporal  pole,  BA38),  (e)  bilateral  middle  temporal  gyri,  (f)  left  parietal  lobe  (including  supramarginal 
gyrus),  and  (g)  bilateral  cerebellum.  A  number  of  these  regions  are  displayed  in  Figure  12,  which  shows  a 
single  subject’s  BOLD  activation  map  for  Task  2. 


As  for  Task  1,  the  regions  targeted  by  Task  2  also  have  significance  for  emotion  processing.  Further,  it  is 
interesting  to  note  that  common  regions  of  activation  were  shared  across  Tasks  1  and  2,  including  structures 
in  the  MNS  (i.e.,  inferior  frontal  gyrus,  superior  temporal  gyrus,  and  parietal  lobe).  These  findings  suggest 
that  shared  neural  mechanisms  exist  between  the  production  and  recognition  of  vocal  emotional  expressions, 
consistent  with  the  findings  from  a  previous  study  which  showed  activation  in  the  inferior  frontal  gyrus 
(specifically)  when  participants  passively  listened  to  emotional  vocal  expressions  and  produced  their  own 
(Aziz-Zadeh,  Sheng,  &  Gheytanchi,  2010).  The  implications  of  these  findings  are  discussed  in  more  detail 
below  (see  General  Discussion). 


Figure  12.  Activation  map  of  axial  slices  from  a  single  subject  showing  contrast  in  BOLD  signal  acquired  during  the  Emotion  Condition  versus 
Neutral  Condition  in  Task  2  (t>3. 17,  p<0.001).  In  this  subject,  several  regions  of  significantly  greater  neuronal  activation  were  associated  with  the 
recognition  of  vocal  emotion  expressions,  including:  a)  superior  frontal  gyrus,  b)  right  middle  frontal  gyrus,  c)  left  middle  frontal  gyrus,  d)  right 
precuneus  (parietal  lobe),  e)  left  supramarginal  gyrus  (parietal  lobe),  f)  right  inferior  frontal  gyrus,  g)  left  inferior  frontal  gyrus,  h)  posterior 
cingulate,  i)  right  middle  temporal  gyrus,  j)  left  middle  temporal  gyrus,  k)  right  cerebellum,  1)  left  cerebellum. 
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Tasks  1  and  2:  General  Discussion 


Our  functional  neuroimaging  paradigm  was  designed  to  map  brain  systems  that  underpin  social  communication 
in  the  BAP.  Specifically,  we  were  interested  in  identifying  systems  that  underlie  the  production  and  recognition 
of  emotional  vocal  expressions,  with  the  view  of  identifying  neurobiological  markers  associated  with  the 
psychoacoustic  endophenotype  of  the  BAP. 

Overall,  our  findings  to  date  across  Tasks  1  and  2  suggest  that  our  functional  neuroimaging  paradigm  has 
successfully  targeted  such  systems,  including  limbic  structures  and  the  mirror  neuron  system  (MNS), 
comprising  premotor  (e.g.,  inferior  frontal  gyrus),  temporal  (e.g.,  superior  temporal  gyrus)  and  parietal  regions 
of  the  brain  (Iacoboni  &  Dapretto,  2006).  These  findings  represent  an  important  scientific  breakthrough,  as  it  is 
the  first  time  such  a  paradigm  has  been  developed  for  use  in  individuals  with  the  BAP.  Previous  research  in 
healthy  individuals  has  shown  that  the  MNS  includes  a  network  of  neurons  that  are  commonly  activated  when 
individuals  perform  a  specific  action  and  when  they  observe  others  performing  the  same  action  (Rizzolatti  & 
Craighero,  2004).  Researchers  have  therefore  proposed  that  the  MNS  underlies  imitative  behavior,  which  in 
turn,  is  crucial  for  the  development  of  social  skills  such  as  understanding  the  intentions  and  emotions  of  other 
people  (Carr,  Iacoboni,  Dubeau,  Mazziotta,  &  Lenzi,  2003;  Rizzolatti  &  Craighero,  2005).  Accordingly,  MNS 
dysfunction  has  been  linked  with  imitation  deficits,  reduced  empathy  and  greater  symptom  severity  in  ASD, 
and  may  thus  be  associated  with  similar  features  in  the  BAP  (Dapretto  et  al.,  2005;  Hadjikhani,  Joseph,  Snyder, 
&  Tager-Flusberg,  2005;  Oberman  &  Ramachandran,  2007;  Yamasaki  et  al.,  2010). 

The  efficacy  of  our  paradigm  is  further  reflected  in  evidence  for  shared  neural  mechanisms  between  the 
production  (Task  1)  and  recognition  (Task  2)  of  emotions  expressed  vocally.  These  common  regions  of 
activation  include  structures  belonging  to  the  classic  MNS  network  (i.e.,  inferior  frontal  gyrus,  superior 
temporal  cortex,  parietal  lobe)  as  well  as  other  regions  external  to  this  network  (i.e.,  superior  frontal  gyrus, 
middle  temporal  gyrus,  insula,  cerebellum).  Of  note,  these  latter  regions  have  also  been  implicated  in  emotional 
processes,  such  as  emotion  regulation,  emotion  recognition,  and  emotional  recall/imagery  (Beauregard,  et  al., 
2001;  Eippert,  et  al.,  2007;  Phan,  et  al.,  2002;  Schutter  &  Honk,  2009).  Our  findings  may  therefore  reflect 
simulation  processes  for  emotional  understanding,  whereby  individuals  use  internal  representations  of  how  they 
express  vocal  emotions  to  understand  the  vocal  emotions  expressed  by  others  (Aziz-Zadeh,  et  al.,  2010; 
Gallese,  2007).  Accordingly,  our  neuroimaging  paradigm  provides  an  exciting  new  technique  for  examining 
differential  patterns  of  activation  during  the  production  and  recognition  of  vocal  emotions  in  individuals  with 
and  without  the  BAP. 

Finally,  as  described  in  our  initial  project  proposal,  examining  MNS  function  in  the  BAP  provides  important 
insight  into  the  neurobiological  and  genetic  bases  of  social  behavior  in  ASD.  To  this  end,  we  are  also  working 
towards  finalizing  our  analyses  of  larger  sets  of  genetic,  behavioral  and  functional  neuroimaging  data,  which 
we  will  combine  to  provide  conclusive  evidence  of  MNS  dysfunction  in  the  BAP. 
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KEY  RESEARCH  ACCOMPLISHMENTS 


The  information  below  summarizes  the  key  research  accomplishments  to  date: 

•  IRB  approval  of  study  and  successful  ongoing  IRB  review  of  study  across  2012-2014. 

•  Systematic  development  and  pilot  testing  of  the  behavioral  and  neuroimaging  protocols 
(Experiments  1  and  2),  which  includes  the  development  of  a  new  piece  of  computer  software  for 
the  collection  of  the  behavioral  data,  as  well  as  the  development  of  a  novel  fMRI  protocol  for  the 
collection  of  in-scanner  behavioral  and  brain  imaging  data. 

•  Ongoing  training  of  the  Research  Assistant  in  paradigm  development,  data  collection  and  data 
analysis  for  Experiments  1  and  2. 

•  Development  of  novel  computational  algorithms  for  analysis  of  voice  recordings  of  BAP  and 
control  participants  to  characterize  the  psychoacoustic  endophenotype  of  the  BAP. 

•  Discovery  of  activation  within  the  mirror  neuron  system,  limbic  system  and  other  regions  associated 
with  vocal  emotion  processing,  as  revealed  through  analysis  of  our  fMRI  data 

•  Discovery  of  shared  neural  mechanisms  between  the  production  and  recognition  of  nonverbal  vocal 
emotional  expressions. 

•  Detailed  characterisation  of  the  neurocognitive  profile  of  the  Broader  Autism  Phenotype  (BAP) 
in  family  members  of  large  families  with  multiple  individuals  affected  with  Autism  Spectrum 
Disorders  (ASD). 

•  Research  presented  at  five  scientific  meetings  between  2013-2014. 

•  Findings  of  Experiments  1  and  2  to  be  published  in  high  impact,  international  scientific  journals 
within  the  next  12  months. 

•  Presentation  of  findings  from  Experiments  1  and  2  to  be  disseminated  at  relevant  community 
and  scientific  meetings  over  the  next  12  months,  including  the  International  Meeting  For 
Autism  Research  (IMFAR). 
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CONCLUSION 


Overall,  this  project  has  been  extremely  successful  and  advanced  the  research  field  in  a  number  of  ways. 
First,  it  has  led  to  the  development  of  new  online  computer  software  to  collect  and  analyze  nonverbal  vocal 
recordings  sensitive  to  the  BAP.  These  new  techniques  can  now  be  applied  to,  and  thus  will  facilitate  the 
collection  of  large-scale  behavioral  data  from  individuals  around  the  globe.  Since  such  techniques  have  not 
previously  existed,  they  represent  a  significant  advance  for  the  field.  Second,  the  study  has  led  to  the 
development  of  a  new  neuroimaging  protocol  capable  of  assessing  brain  function  and  structure,  particularly 
social  and  emotional  functioning,  in  individuals  with  the  BAP  and  ASD.  Our  findings  to  date  indicate  that 
our  functional  neuroimaging  paradigm  can  be  used  to  target  regions  within  the  mirror  neuron  system  and 
limbic  system  of  the  brain,  augmenting  our  understanding  of  the  neurobiology  of  the  BAP.  Overall,  such 
techniques  had  not  previously  been  used  with  individuals  with  the  BAP  and  thus,  represent  an  important 
advance  for  the  field.  Combined,  the  techniques  now  provide  new  and  exciting  ways  for  objectively  assessing 
core  features  of  autism  that  have  eluded  the  field  to  date.  This  will  ultimately  provide  important  new  avenues 
for  identifying  genes  underpinning  ASD  and  the  BAP. 


We  would  like  to  thank  the  US  Department  of  Defense  for  their  generous  support  of  this  research  and  will 
ensure  that  this  support  is  appropriately  recognized  and  acknowledged  in  all  publications,  presentations  and 
other  outcomes  arising  from  this  research. 
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PUBLICATIONS,  ABSmAlfTS,  4^1il9BBfiENTATIONS 


Preliminary  aspects  of  the  research  was  presented  at  five  scientific  meetings  between  2013-2014,  as 
follows: 

1)  Verbal  Pragmatics  and  Vocal  Emotion  Processing  in  the  Broader  Autism  Phenotype.  Oral 
presentation,  Neurocognition  &  Neuroimaging  Laboratory,  The  University  of  Melbourne  (March, 
2013). 

2)  Verbal  Pragmatics  and  Vocal  Emotion  Processing  in  the  Broader  Autism  Phenotype.  Oral 
presentation,  Collaborative  Autism  Team  Study,  Annual  Autism  Retreat,  The  University  of 
Melbourne.  This  retreat  is  well  attended  by  researchers  from  The  University  of  Melbourne,  the 
Royal  Children’s  Hospital,  the  Melbourne  Brain  Centre,  and  the  Florey  Institute  of  Neuroscience 
and  Mental  Health  (May,  2013  and  November  2014). 

3)  Verbal  Pragmatics  and  Vocal  Emotion  Processing  in  the  Broader  Autism  Phenotype.  Platform 
presentation,  Neurocognition  &  Neuroimaging  Workshop,  2013  Epilepsy  Research  Retreat, 
Melbourne  Brain  Centre,  Florey  Institute  of  Neuroscience  &  Mental  Health,  The  University  of 
Melbourne,  Australia  (July  2013). 

4)  Vocal  Emotion  Processing  in  the  Broader  Autism  Phenotype.  Platform  presentation,  School  of 
Psychological  Sciences,  The  University  of  Melbourne,  Australia  (November  2014). 
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P41EMTS^lIiLlGBMl3|S 


Nothing  to  report. 
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REPORTABLE  OUTCOMES 


•  Development  of  a  novel  web-based  paradigm  to  collect  extensive  data  sets  of  vocal  responses  across 
tasks  of  spontaneous  production,  recognition  and  imitation  of  nonverbal  emotional  vocal 
expressions. 

•  Development  of  a  novel  functional  neuroimaging  paradigm  to  target  the  mirror  neuron  system  and 
other  brain  regions  of  significance  to  social  and  emotional  processing  in  the  BAP  and  ASD. 
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OTHER  ACHIEVEMENTS 


Nothing  to  report. 
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